Search for: All records

Preferences within a group of people are not uniform but follow a distribution. While existing alignment methods like Direct Preference Optimization (DPO) attempt to steer models to reflect human preferences, they struggle to capture the distributional pluralistic preferences within a group. These methods often skew toward dominant preferences, overlooking the diversity of opinions, especially when conflicting preferences arise. To address this issue, we propose Group Distributional Preference Optimization (GDPO), a novel framework that aligns language models with the distribution of preferences within a group by incorporating the concept of beliefs that shape individual preferences. GDPO calibrates a language model using statistical estimation of the group's belief distribution and aligns the model with belief-conditioned preferences, offering a more inclusive alignment framework than traditional methods. In experiments using both synthetic controllable opinion generation and real-world movie review datasets, we show that DPO fails to align with the targeted belief distributions, while GDPO consistently reduces this alignment gap during training. Additionally, our evaluation metrics demonstrate that GDPO outperforms existing approaches in aligning with group distributional preferences, marking a significant advance in pluralistic alignment. 

Much work on the cultural awareness of large language models (LLMs) focuses on the models' sensitivity to geo-cultural diversity. However, in addition to cross-cultural differences, there also exists common ground across cultures. For instance, a bridal veil in the United States plays a similar cultural-relevant role as a honggaitou in China. In this study, we introduce a benchmark dataset CUNIT for evaluating decoder-only LLMs in understanding the cultural unity of concepts. Specifically, CUNIT consists of 1,425 evaluation examples building upon 285 traditional cultural-specific concepts across 10 countries. Based on a systematic manual annotation of cultural-relevant features per concept, we calculate the cultural association between any pair of cross-cultural concepts. Built upon this dataset, we design a contrastive matching task to evaluate the LLMs' capability to identify highly associated cross-cultural concept pairs. We evaluate 3 strong LLMs, using 3 popular prompting strategies, under the settings of either giving all extracted concept features or no features at all on CUNIT Interestingly, we find that cultural associations across countries regarding clothing concepts largely differ from food. Our analysis shows that LLMs are still limited to capturing cross-cultural associations between concepts compared to humans. Moreover, geo-cultural proximity shows a weak influence on model performance in capturing cross-cultural associations. 

As part of a larger design-based research study, we developed middle school social studies and computer science units with a focus on food sovereignty to help students understand treaties and their ongoing impact on Indigenous people today. This paper reports on the iterative process of developing and implementing culturally responsive-sustaining computing curricula, particularly focusing on feedback from a master teacher panel. Teachers were excited about engaging students in computer science by linking it to their everyday lives, but also concerned about the difficulty of making complex concepts like food sovereignty accessible to students. 

We designed and implemented a data visualization project in a seventh-grade classroom focusing on the loss of tribal lands in Montana, United States of America. We aim to understand how students engaged with various scaffolds and how those scaffolds supported them in critically engaging with historical data through making hands-on projects. We analyzed data from multiple sources, including classroom implementation transcripts, student-created artifacts, and pre- and post-surveys. We observed that while students enjoyed creating data visualizations, they struggled to interpret them within their historical context despite the provision of multiple forms of scaffolds. We believe it is important to design a system of scaffolds to further support students in critically engaging with historical data and in developing critical data literacy.